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Abstract 

We describe quantization designs which lead to asymptotically and order optimal functional 
quantizers. Regular variation of the eigenvalues of the covariance operator plays a crucial role 
to achieve these rates. For the development of a constructive quantization scheme we rely on 
the knowledge of the eigenvectors of the covariance operator in order to transform the problem 
into a finite dimensional quantization problem of normal distributions. 
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> : 

r — 1 1 Introduction 

• Functional quantization of stochastic processes can be seen as a discretization of the path-space 
| of a process and the approximation (coding) of a process by finitely many deterministic functions 
' from its path-space. In a Hilbert space setting this reads as follows. 

Let (H,< •, • >) be a separable Hilbert space with norm || • || and let X : (Q,A, IP) — > H be 
a random vector taking its values in H with distribution Px- For n 6 N, the L 2 -quantization 
problem for X of level n (or of nat-level log n) consists in minimizing 
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over all subsets a C H with card(a) < n. Such a set a is called n-codebook or n-quantizer. The 
minimal nth quantization error of X is then defined by 

eJX) :=inf|(Emin||X-a|| 2 ) 1/2 : a C H, card(a) < n\ . (1.1) 

Under the integrability condition 

E||X|| 2 <cx) (1.2) 

the quantity e n (X) is finite. 



*This work was supported in part by the AMaMeF Exchange Grant 1323 of the ESF. 
tUniversitat Trier, FB IV-Mathematik, D-54286 Trier, Germany. E-mail: luschgy@uni-trier.de 
"•■Laboratoire de Probabilites et Modeles aleatoires, UMR 7599, Universite Paris 6, case 188, 4, pi. Jussieu, F-75252 
Paris Cedex 5. E-mail:gpa@ccr . jussieu.fr 

'Universitat Trier, FB IV-Mathematik, D-54286 Trier, Germany. E-mail: wilbertz@uni-trier.de 



1 



For a given n-quantizer a one defines an associated closest neighbour projection 

vr a := ^al Ca(Q) 

and the induced a-quantization (Voronoi quantization) of X by 

X a :=ir a (X), (1.3) 
where {C a (a) : a G a} is a Voronoi partition induced by a, that is a Borel partition of H satisfying 

C a (a) C V a (a) := {x £ H : \\x - a\\ = min \\x - b\\} (1.4) 

for every a G a. Then one easily checks that, for any random vector X' : — > a C , 

E ||X - A|| 2 > E ||X - A a || 2 = E min \\X - a|| 2 

so that finally 

e n {X) = inf j(E||A - X|| 2 ) 1/2 : X = f(X)J : H ^ H Bovel measurable, (1.5) 
card(/(il)) < n} 

= inf j(E||A - XII 2 ) 1 / 2 : X : Q -> H random vector, card(X(fi)) < n| . 

Observe that the Voronoi cells V a (a) ,a £ a are closed and convex (where convexity is a charac- 
teristic feature of the underlying Hilbert structure). Note further that there are infinitely many 
a-quantizations of X which all produce the same quantization error and X a is P-a.s. uniquely 
defined if Px vanishes on hyper planes. 

A typical setting for functional quantization is H = £ 2 ([0, 1], dt) but is obviously not restricted 
to the Hilbert space setting. Functional quantization is the natural extension to stochastic processes 
of the so-called optimal vector quantization of random vectors in H = M. d which has been extensively 
investigated since the late 1940's in Signal processing and Information Theory (see [4], [7]). For the 
mathematical aspects of vector quantization in M. d , one may consult [5], for algorithmic aspects see 
[16] and "non-classical" applications can be found in [14], [15]. For a first promising application 
of functional quantization to the pricing of financial derivatives through numerical integration on 
path-spaces see [17]. 

We address the issue of high-resolution quantization which concerns the performance of n- 
quantizers and the behaviour of e n (X) as n — > oo. The asymptotics of e n (X) for M. d - valued random 
vectors has been completely elucidated for non-singular distributions Px by the Zador Theorem 
(see [5]) and for a class of self-similar (singular) distributions by [6]. In infinite dimensions no such 
global results hold, even for Gaussian processes. 

It is convenient to use the symbols ~ and ~, where a n ~ b n means a n /b n — ► 1 and a n ~ b n 
means limsup,^^ a n /b n < 1. A measurable function if : (s,oo) — > (0, oo) (s > 0) is said to be 
regularly varying at infinity with index 6 G M if, for every c > 0, 

,. f( cx ) b 
lim — — - = c . 

Now let X be centered Gaussian. Denote by Kx C H the reproducing kernel Hilbert space 
(Cameron-Martin space) associated to the covariance operator 

C x :H^H, C x y :=E(<y,X>X) (1.6) 
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of X. Let Ai > A2 > • • • > be the ordered nonzero eigenvalues of C x and let {uj : j > 1} be the 
corresponding orthonormal basis of supp(Px) consisting of eigenvectors (Karhunen-Loeve basis). 

( d \ 

If d := A\m.Kx < 00, then e n (X) = e n I M(0,Xj) , the minimal nth L 2 -quantization error of 

V=i / 

d 

(£) A/"(0, Xj) with respect to the ^-norm on M. d , and thus we can read off the asymptotic behaviour 
j'=i 

of e n (X) from the high-resolution formula 



d / x i/2d //7-u 2\ ( d + 2 )/ 4 

e n (0JV*(O,A j ))~?(d)v^i : (nJ =1 A J -) (^-j ""^asn^oo (1.7) 



where g(d) G (0, 00) is a constant depending only on the dimension d (see [5]). Except in dimension 
d = 1 and d = 2, the true value of g(ci) is unknown. However, one knows (see [5]) that 

[A x 1/2 



g(d) ~ as d^oo. (1.8) 

\27re / 

Assume dimKx = 00. Under regular behaviour of the eigenvalues the sharp asymptotics of 
e n (X) can be derived analogously to (1.7). In view of (1.8) it is reasonable to expect that the 
limiting constants can be evaluated. The recent high-resolution formula is as follows. 

Theorem 1. ([11]) Let X be a centered Gaussian. Assume Xj ~ (p(j) as j — > 00, where if : 
(s, 00) — > (0, 00) is a decreasing, regularly varying function at infinity of index —b<—l for some 
s > 0. Set, for every x > s, 

V>(x) := 

Then 

en(X) ~ ( ( ) \ iP{\ogn)- l l 2 as n ^ 00. 




A high-resolution formula in case h = 1 is also available (see [11]). Note that the restriction 

00 

— b < — 1 on the index of 92 is natural since Yl -\j < 00 • The minimal L r -quantization errors of 

X, < r < 00, are strongly equivalent to the L 2 -errors e n {X) (see [3]) and thus exhibit the same 
high-resolution behaviour. 

The paper is organized as follows. In Section 2 we investigate a quantization design, which 
furnishes asymptotically optimal quantizers in the situation of Theorem 1. Here the Karhunen- 
Loeve expansion plays a crucial role. In section 3 we state different quantization designs, which are 
all at least order-optimal and discuss their possible implementations regarding the example of the 
Brownian motion. The main focus in that section lies on "good" designs for finite n G N. 

2 Asymptotically optimal functional quantizers 

Let X be a if-valued random vector satisfying (1.2). For every n G N, L 2 -optimal n-quantizers 
a C H exist, that is 

(E min||X-a|| 2 ) 1/2 = e n {X). 
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If card (supp(Px)) > n , optimal n-quantizers a satisfy card(a) = n, ¥(X G C a (a)) > and the 
stationarity condition 

o = E(X|{XeC„(a)}),oea (2.1) 

or what is the same 

X a =E(X | X a ) (2.2) 

for every Voronoi partition {C a (a) : a G a} (see [10]). In particular, El a = ~EX. 

Now let X be centered Gaussian with dimi^x = oo. The Karhunen-Loeve basis {uj : j > 1} 
consisting of normalized eigenvectors of C x is optimal for the quantization of Gaussian random 
vectors (see [10]). So we start with the Karhunen-Loeve expansion 

oo 

x^xy% Uj , 

1 /2 

where £j =< X,uj > j\- ,j > 1 are i.i.d. A/(0, l)-distributed random variables. The design of 
an asymptotically optimal quantization of X is based on optimal quantizing blocks of coefficients 
of variable (n-dependent) block length. Let n G N and fix temporarily m, I, n\, . . . , n m G N with 
W^ =1 rij < n, where m denotes the number of blocks, I the block length and rij the size of the 
quantizer for the jth block 

Let ttj C M. 1 be an L 2 -optimal rij-quantizer for an d let = J be a <x,-quantization of 
(Quantization of blocks instead of (A^ 1 ^ +1 ^_ 1 )/ +1 , . . . , ^{ 2 ^ji) is asymptotically good 
enough. For finite n the quantization scheme will be considerably improved in Section 3. ) Then, 
define a quantized version of X by 

m I 

i "-EE A H) W ^ 1 )^(H)W (2-3) 

j=l k=l 

It is clear that card(V n (ft)) < n. Using (2.2) for one gets El" = 0. If 
then 

aeX^jQj jr' = l fc=l 

where a = (a^, . . . , a/ m )) G y.™ =1 atj. Observe that in general, X n is not a Voronoi quantization of 
X since it is based on the (less complicated) Voronoi partitions for < m. (X n is a Voronoi 

quantization if I = 1 or if A( J _ 1 ^ + i = . . . = Xji for every j.) Using again (2.2) for ^ and the 
independence structure, one checks that X n satisfies a kind of stationarity equation: 

E(X | X n ) = X n . 

Lemma 1. Let n > 1. For every I > 1 anc? every m > 1 

E||X-X"|| 2 <^A (j _ 1)m e n .(Ar(0,70) 2 + E V ( 2 - 4 ) 

jr' = l j>ml+l 

Furthermore, (2.4) stands as an equality if I = 1 (or A(j_!^ +1 = . . . = Ajj /or every j, / > lj. 
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Proof. The claim follows from the orthonormality of the basis {uj : j > 1}. We have 

ml , ., 

E||x-xi 2 = E E \j-i)i+k^ I - I 2 + £ A, 

j=l k=l j>rnl+l 
m I . 

< E E E I - I 2 + E a, 

j=l k=l j>ml+l 

m 

= EA(M) i+ ie„ 3 (^) 2 + E A,, 

j'=l j>ml+l 



□ 



Set 



C(Z) :=supA; 2 /'e fc (7V(0,70) 2 - (2-5) 
fc>i 



By (1.7), C(Z) < oo. For every I G N, 

e„ 3 (iV(0,/0 2 <^ 2/Z C(0 (2.6) 

Then one may replace the optimization problem which consists, for fixed n, in minimizing the right 
hand side of Lemma 1 by the following optimal allocation problem: 

m 

min{C(/) A (j _ 1)m nT 2/ ' + Xj : m,l,m, . . . ,n m e N,Uf =1 nj < n}. (2.7) 

j=l j>ml+l 

Set 

m = m(n, I) := max{fc > 1 : n 1 ^ \ l ff_ 1)l+1 (U^ =1 \ {j _ 1)l+1 )- l / 2k > 1}, (2.8) 

n, =n,(n,/) := [n 1 ^^^ iA^dj+x)-'/ 2 ™], J € {l,...,m}, (2.9) 
where [x] denotes the integer part of x G 1R and 

l = l n := [(max{l,logn})*], i? G (0,1). (2.10) 

In the following theorem it is demonstrated that this choice is at least asymptotically optimal 
provided the eigenvalues are regularly varying. 

Theorem 2. Assume the situation of Theorem 1. Consider X n with tuning parameters defined 
in (2.8)-(2.10). Then X n is asymptotically n-optimal, i.e. 

{¥,\\X - X n \\ 2 f' 2 ^ e n {X) as n -> oo. 

Note that no block quantizer with fixed block length is asymptotically optimal (see [11]). As 
mentioned above, X n is not a Voronoi quantization of X. If a n := X n (Q), then the Voronoi 
quantization X an is clearly also asymptotically n-optimal. 

The key property for the proof is the following ^-asymptotics of the constants C(l) defined 
in (2.5). It is interesting to consider also the smaller constants 



Q(l) := lim k 2 ' l e k {M{0,Ii)) 2 (2.11) 

k^oo 



(see (1.7)). 
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Proposition 1. The sequences (C(Z))/>i and (Q(l))i>\ satisfy 

llm ^ =llm M = mt M = mf M = 1 . 

Z^oo I Z^oo I 1>1 I Z>1 I 



Proof. From [11] it is known that 



liminf^ = l. (2.12) 

Z^oo I 



Furthermore, it follows immediately from (1.7) and (1.8) that 

Jim 9^1 = i. (2.13) 

I— »oo I 

(The proof of the existence of lim C(l)/l we owe to S. Dereich.) For Iq, I € N with Z > Iq, write 

Z^oo 

I = nl$ + m with n € N, m € {0, . . . , Iq — 1}. 

Since for every fceN, 

one obtains by a block-quantizer design consisting of n blocks of length l and m blocks of length 
1 for quantizing iV(0, ij), 

e fc (AA(0,/0) 2 < ne [fc;o/i] (AA(0,/ /o )) 2 + me [fel/i] (AA(0,l)) 2 . (2.14) 

This implies 



fc 2/Z k 2/l 

C{1) < nC(l ) sup — r-Tjr^jr- + mC(l) sup ■ 



k>T [ifeV'p/'o — ^^>i [A;V*]2 
< 4 1/io nC(Z ) +4mC(l). 
Consequently, using ra/Z < 1/Zo, 

C(Z) < 4 1 / / pC(/ ) 4mC(l) 



and hence 
This yields 

It follows from (2.14) that 
Consequently 

and therefore 



I ~ l Q I 



C(Z) 4VfeC(Zo) 
hm sup — - — < 



limsup^<liminf^ = l. (2.15) 
Q(l) < nQ(l )+mQ(l). 



Q(l) Q(/p) mQ(l) 
Z " Z i 

1=lim M<«M. 

Z— >oo Z Zn 
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This implies 

inf 9M = 1 _ (2.16) 

lo>l l 

Since Q(l) < C(l), the proof is complete. □ 

The n-asymptotics of the number m(n, l n )l n of quantized coefficients in the Karhunen-Loeve 
expansion in the quantization X n is as follows. 

Lemma 2. ([12], Lemma 4-8) Assume the situation of Theorem 1. Let m(n,l n ) be defined by (2.8) 
and (2.10). Then 

I , x, 21ogn 
m(n, l n )l n ~ — ; — as n — > oo. 
o 

Proof of Theorem 2. For every n G N, 



< 4 1 / i mn- 2 / m, (n™ 1 A (j _ 1)1+1 ) 1 / m 

< 4}' l m\ m _ 1)l+l . 



Therefore, by Lemma 1 and (2.6), 

,C(l) 



E\\X-X n \\ 2 <^ l ^ml\ (m _ 1)l+1 + Yl X i 

j>ml+l 

for every n G N. By Lemma 2, we have 

i ^ i \7 21ogn 
m/ = m{n,l n )l n ~ — - — as n — > oo. 

Consequently, using regular variation at infinity with index — b < — 1 of the function 99, 



and 

E mlip(ml) 1 / 2\ 1-6 x 
Xj — — - I - 1 V(logn) as n 00, 

j>ml+l ^ ' 

where, like in Theorem 1, tp(x) = l/x<p(x). Since by Proposition 1, 

Urn ^"f""' = 1, 

n— >oo /_ 



2A 1 - 6 6 



one concludes 

E||X-X n || 2 ~ ^y^(logn)^ 1 as n -»• 00. 

The assertion follows from Theorem 1. □ 
Let us briefly comment on the true dimension of the problem. 

For n E N, let C n (X) be the (nonempty) set of all L 2 -optimal n-quantizers. We introduce the 
integral number 

d* n (X) := min{dim span (a) : a G C n (X)} . (2.17) 
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It represents the dimension at level n of the functional quantization problem for X. Here span(a) 
denotes the linear subspace of H spanned by a. In view of Lemma 2, a reasonable conjecture for 
Gaussian random vectors is d^(X) ~ 21ogn/6 in regular cases, where —b is the regularity index. 
We have at least the following lower estimate in the Gaussian case. 

Proposition 2. Assume the situation of Theorem 1. Then 

a*(y\ > 1 21ogn 

d nW ~ £7(6=T) — — as n - °°- 

Proof. For every n G N, we have 

fc 



d* (X) = min < 

(see [10]). Define 



&>0:e n ((g)iV(0,A i )) 2 + £ A j < e n (X) 5 
j=i i>fc+i 



(2.18) 



c n := mm 

Clearly, c n increases to infinity as n — > oo and by (2.18), c n < d* n {X) for every n G N. Using 
Theorem 1 and the fact that -ip is regularly varying at infinity with index b — 1, we obtain 

((ft-lMcv,))- 1 - E A,- ~ enpQ 2 ~ (HV '-A-^logn)- 1 

j>Cn + l ^ ' 

and thus 

,f ^ /2V- 6 1 /n . . ( 1 21ogn\ 

7 T ^(logn) ~ V rr77ft=T1 — : — as n ^ oo. 



Consequently, 

1 2 log n 

c ™ ~ W^) ~b~ as n °°- 

This yields the assertion. □ 

3 Quantizer designs and applications 

In this section we are no longer interested in only asymptotically optimal quantizers of a Gaussian 
process X, but rather in really optimal or at least locally optimal quantizers for finite n G N. 

As soon as the Karhunen-Loeve basis (uj)j>± and the corresponding eigenvalues (Xj)j>i of the 
Gaussian process X are known, it is possible to transform the quantization problem of X in H into 
the quantization of (£) ( ?L 1 J\f(0, Aj) on I 2 by the isometry S : H —> I 2 

X l-> (< Uj,X >)_,->! • 

and its inverse 

S- 1 : (/ 2 ,< • > K ) - (H,< .,. >), l»J2 l i u r (3- 1 ) 

j>i 

The transformed problem then allows as we will see later on a direct access by vector quantization 
methods. 

The following result is straightforward. 
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Proposition 3. Denote by a C H an arbitrary quantizer for X with associated Voronoi quantiza- 
tion X a and Voronoi partition {C a (a) : a £ a}. If 

S:(H,<;->)^(K,<;-> K ) 

is a bijective isometry from H — ► K , where K is another separable Hilbert space, i.e. S is linear 
and \\Sx — Sy\\p = \\x — y\\ for every x,y £ H , then 

1. S(C a (a)) = Csa(Sa), for every a£ a 

^ — - — .Sa 

2. S(X a ) = S(X) is a Voronoi quantization of S(X) induced by Sa 

3. Emin aGa ||X - a\\ 2 = Emm S aeSa\\S(X) - Sa\\ 2 K 

Consequently we may focus on the quantization problem of the Gaussian random vector 

C ■■= s(x) 

on I 2 with distribution 

oo 

C=(0),>i~(g)AA(0,A J ) 
i=i 

for the eigenvalues (Xj)j>i of Cx- Note, that in this case (Xj)j>i also become the eigenvalues of 
the covariance operator C^. 

3.1 Optimal Quantization of (g^-A^O, Xj) 

Since an infinite dimensional quantization problem is without any modification not solvable by a 
finite computer algorithm, we have to somehow reduce the dimension of the problem. 

Assume a to be an optimal n-quantizer for (g)j? =1 A/'(0, Xj), then U := span(a) is a subspace of 
I 2 with dimension d* = dim U < n — 1. Consequently there exist djj orthonormal vectors in I 2 such 
that span(«i, . . . , u^* ) = U. 

Theorem 3.1 in [10] now states, that this orthonormal basis of U can be constructed by eigen- 
vectors of C{, which correspond to the d* n largest eigenvalues. To be more precise, we get 

4((g)AA(0,A,)) =e 2 (^M(0,Xn))+ £ Xj. (3.2) 

Hence it is sufficient to quantize only the finite-dimensional product measure ®"=!^V(0, Xj) and 
to fill the remaining quantizer components with zeros. 

Therefore we denote by ( d the projection of ( = (Cj)j>i on the first (i-components, i.e. ( d = 
(Ci-.-.-Q). 

This approach leads for some d G N to our first quantizer design. 
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Quantizer Design I Product Quantizer for §Q°° =1 A/"(0, Xj) 



Require: Optimal ®^ =1 M(0, Xj )-Quantizer a d C M. d with card(a d ) < n 
Quantizer: 



Quantization: 



Distortion: 



a 1 ■= a d x {0} x . . . 

r 7 = ^oi Ca(Q , ) (o = (c iad ,o,...) 



E||C-C Q/ ||f 2 =4((8)AA(0,A,))+ £ Aj 



The claim about the distortion of £ a becomes immediately evident from the orthogonality of 
the basis Vj = (<5jj)j>i in I 2 and 



E||C-C Q 11/2 = E 



3=1 j>d+l 
d 



2 



2 



i=l j>d+l 



Unfortunately the true value of ci* is only known for n = 2, which yields d% = 1, but from 
Proposition 2 we have the lower asymptotical bound 

1 21ogn 

< d„, as n -> oo, 



6V(b-i) 5 



whereas there is a conjecture for it to be d* ~ 21ogn/6. 

A numerical approach for this optimal design by means of a stochastic gradient method will be 
introduced in section 3.2, where also some choices for the block size d with regard to the quantizer 
size n will be given. 

In addition to this direct quantization design, we want to present some product quantizer designs 
for 0°^=! AA(0, Aj), which are even tractable by deterministic integration methods and therefore 
achieve a higher numerical accuracy and stationarity. These product designs reduce furthermore 
the storage demand for the precomputed quantizers when using functional quantization as cubature 
formulae e.g. 

To proceed this way, we replace the single quantizer block a d from Quantizer Design I by the 
cartesian product of say m smaller blocks with maximal dimension I < d. We will refer to the 
dimension of these blocks also as the block length. 

Let U denote the length of the i-th block and set 

i-l 



ki := 0, ki := y*J v , ie{2,...,m}, 



u=l 

-d 



then we obtain a decomposition of £ into 

C d = (C (1) ,-..,C (m) ), with C (l) :=(C fcl+ i,...,C^=C fcl+1 ). (3.3) 
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So we state for some I 6 N: 



Quantizer Design II Product Quantizer for Af(0, Xj) 



Require: Optimal ®j=fc +1 N(Q, Aj)-Quantizers aW c R J< with card(o;W) < n» for some Integers 

m G N, ii, . . . Z m < Z, ni, . . . , n m > 1, J]^i «j < « solving 
Block Allocation: 



Quantizer: 



Quantization: 



Distortion: 



£4 ® + £ A, ^min. 

.1=1 \fc=fc l +l / j>fc m+ i+l J 



a// : =n« w x {o} x ... 



i=i 



Q (l) a (m) 



C = £ «lo a («»)(0 = (C« ■ ■■■,c< m > ,0,...) 



i=l Vj^+l / j>k m+ i+l 



Note that we do not use the asymptotically block allocation rules for the rij from (2.9), but 
perform instead the block allocation directly on the true distortion of the quantizer block and not 
on an estimate for them. 

Next, we weaken our quantizer design, and obtain this way the asymptotically optimal design 
from Theorem 2. 

In fact the quantizer used for this scheme are a little bit more universal, since they do not 
depend on the position of the block, but not at all more simply to generate. 

The idea is to quantize blocks £W ~ A/"(0, Ii { ) of standard normals £ = (£j)j>i ~ ^^ =1 /\f(0, 1) 
and to weight the quantizers by 

: = (yx^+i, X ki+1 ^j , i€{l,...,m}, 

that is 

VAW ®a W = • • • ,^k i+1 a ki+1 ) ■ a = (a ki+1 , . . . ,a ki+1 ) £ a W }. 

The design for some I £ N then reads as follows: 
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Quantizer Design III Product Quantizer for <^> ( ^ =1 M(0, Xj) 

Require: Optimal ®j=fc +1 -^(0, 1)-Quantizers a® C IR/ 1 with card(aW) < n« for some Integers 

m £ N, l\, . . . l m <l, n 1 ,...,n m >l, n^=i <n solving 
Block Allocation: 

£ E ^ ). + E A,U min . 

ki=ij=ki+i v v j>fc m+ i+i J 

Quantizer: 

TO 

a m := Yl 7a« ® aW x {0} x . . . 

i=i 

Quantization: 

TO 

a=(a( 1 ),...,a(™),0,...)e« I " i=1 ° 

Distortion: 

TO y (j) \ 2 

EiK-c a/// iif 2 = E E VEK,--(*» a ).) + E ^ 

j=ij=fe i +i v J/ i>fc m+1 +i 



In the end we state explicitly the case I = 1, for which the Designs II and III coincide, and which 
relies only on one dimensional quantizers of the standard normal distribution. These quantizers can 
be very easily constructed by a standard Newton-algorithm, since the Voronoi-cells in dimension 
one are just simple intervals. 

This special case corresponds to a direct quantization of the Karhunen-Loeve expansion (2.1). 

We will refer to this design also as scalar product quantizer. 
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Quantizer Design IV Product Quantizer for M(0, Xj 



Require: Optimal A/"(0, 1)-Quantizers a; C R with card(aj) < rij for some Integers m £ 

N, n 1 ,...,n m >l, n™ i n i < n solving 
Block Allocation: 

( m \ 

EVn J (AA(0,l))+ A„ ^min. 

{j=l j>m+l J 

Quantizer: 

m 



Quantization: 

m 

t aIV = e «n i c4^)(o)=(^r\--->v^u am ,o,...; 

Distortion: 

Eiic-r /v 'ii? 2 = Ev^(^(o,i))+ E ^ 

jr' = l j>m+l 



Clearly, it follows from the decomposition (3.3), that Design I is optimal as soon the quantization 
of (0, A n ) is optimal. Furthermore we obtain the proof of the asymptotically optimality for 

the quantizer Designs II and III from Theorem 2 using the tuning parameter 

I :=l n :=[(max{l,logn}) e } for some 9 £ (0, 1), (3.4) 

i.e. 

E||C - C qI || 2 ~ E||C - (""lip ~ E||C - C aIII \\p ~ (^'"'^-^(logn)- 1 

as n — > oo. 

Using the same estimates as in the proof of Theorem 2 for the Design IV, we only get 

E iic-c^Mr i 4C TV )+ > °^' < 3 - 5 > 

so that we only can state, that Design IV is rate optimal. 

Remark 3.1. Note that if we replace the assumption of optimality for the quantizer blocks by sta- 
tionarity in Designs I-IV, the resulting quantizers are again stationary (but not necessary asymp- 
totically optimal). 

3.2 Numerical optimization of quadratic functional quantization 

Optimization of the (quadratic) quantization of R d -valued random vector has been extensively 
investigated since the early 1950's, first in 1-dimension, then in higher dimension when the cost of 
numerical Monte Carlo simulation was drastically cut down (see [4]). Recent application of optimal 
vector quantization to numerics turned out to be much more demanding in terms of accuracy. In 
that direction, one may cite [16], [13] (mainly focused on numerical optimization of the quadratic 
quantization of normal distributions). To apply the methods developed in these papers, it is 
more convenient to rewrite our optimization problem with respect to the standard ci-dimensional 
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0.224 - 
0.222 - 
0.22 - 
0.218 - 



d(n).2 



d(n) = 



d(n) . 4 



Figure 1: Optimal functional quantization of the Brownian motion, n ^ logn (e n (W, L T )) , n€L 
{6, . . . , 160} for blocksizes d n G {2, 3, 4, 5}. Vertical dashed lines: critical dimensions for d* n , e 2 « 7, 



= 3 ~ 20, e 4 « 55, e 5 « 148. 



distribution A/"(0, 7 d ) by simply considering the Euclidean norm derived from the covariance matrix 
Diag(Ai,...,A d .) i.e. 



(Quantizer Design I) 44> < 



n 



■optimal quantization of (^7^(0, 1) 



k=i 



for the covariance norm \(z±, . . . , z d * )| = X^fc=i -^fc^fc- 

The main point is of course that the dimension d* n is unknown. However (see Figure 1), one 
clearly verifies on small values of n that in the case of the Brownian Motion, i.e b = 2 the conjecture 
(eZ* ~ logn) is most likely true. Then for higher values of n one relies on it to shift from one 
dimension to another following the rule d* n = d, n£ {e d , . . . , e d+1 — 1}. 



3.2.1 A toolbox for quantization optimization: a short overview 

Here is a short overview of stochastic optimization methods to compute optimal or at least locally 
optimal quantizers in finite dimension. For more details we refer to [16] and the references therein. 

Let Z = M(0;ld) and denote by D%(x) the distortion function, which is in fact the squared 
quantization error of a quantizer x G H n = (K d ) n in n-tuple notation, i.e. 

D% : H n — > ft, ihE min \\Z-Xi\\ 2 H . 

l<i<n 

Competitive Learning Vector Quantization (CLVQ). This procedure is a recursive stochastic ap- 
proximation gradient descent based on the integral representation of the gradient V-D^f(x), xG H n 
of the distortion as the expectation of a local gradient and a sequence of i.i.d. random variates, i.e. 

Vx G H n , VL»f (x) = E(Vr>f (x,Z)) and Z k i.i.d., Z 1 =M(0,I d ) 
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for VD%{x) = (2j Ci{x) (xi - OPz(de)) 1<i<n and VD*(x, Z) = {2(x t - Z)l Ci{x) (Z)) ^ n so that, 
starting from x(0)€ (R d ) n , one sets 

V£;>0, x(fc + l) = x(A:)-^-V J D^(x(fc),Z fc+1 ) 

where c G (0, 1] is a real constant to be tuned. As set, this looks quite formal but the operating 
CLVQ procedure consists of two phases at each iteration: 

(i) Competitive Phase: Search of the nearest neighbor ic(/c)i*(fc+i) of Zfc+i among the components 
of x(k)i, i = 1, . . . ,n (using a "winning convention" in case of conflict on the boundary of the 
Voronoi cells). 

(ii) Cooperative Phase: One moves the winning component toward Cfc+i using a dilatation i.e. 
x(k + l)i.( fc +i) = Dilatation Cfc+lil __^_(x(/c) i , (fc+1) ). 

This procedure is useful for small or medium values of n. For an extensive study of this proce- 
dure, which turns out to be singular in the world of recursive stochastic approximation algorithms, 
we refer to [14]. For general background on stochastic approximation, we refer to [8, 1]. 

The randomized "Lloyd I procedure". This is the randomization of the stationarity based fixed 
point procedure since any optimal quantizer satisfies the stationarity property: 

gx(k+i) = E ( Z | £*(fc)) ? ^(o) C U d . 

At every iteration the conditional expectation E(Z | Z x ^) is computed using a Monte Carlo sim- 
ulation. For more details about practical aspects of Lloyd I procedure we refer to [16]. In [13], an 
approach based on genetic evolutionary algorithms is developed. 

For both procedures, one may substitute a sequence of quasi-random numbers to the usual 
pseudo-random sequence. This often speeds up the rate of convergence of the method, although 
this can only be proved (see [9]) for a very specific class of stochastic algorithm (to which CLVQ 
does not belong). 

The most important step to preserve the accuracy of the quantization as n (and d* n ) increase is 
to use the so-called splitting method which finds its origin in the proof of the existence of an optimal 
n-quantizer: once the optimization of a quantization grid of size n is achieved, one specifies the 
starting grid for the size n + 1 or more generally n + v, v > 1, by merging the optimized grid of 
size n resulting from the former procedure with v points sampled independently from the normal 
distribution with probability density proportional to where (p denotes the p.d.f. of M(0; La). 
This rather unexpected choice is motivated by the fact that this distribution provides the lowest 
in average random quantization error (see [2]). 

As a result, to be downloaded on the website [18] devoted to quantization: 

www. quantize .maths-fi . com 

o Optimized stationary codebooks for W: in practice, the n-quantizers a := a d ™ of the distribu- 
tion (g^A^O, A fc ), rc = l up to 10 000 (<2* runs from 1 up to 9). 

o Companion parameters: 

- distribution of W" 1 : P(W~< = x t ) = P(Z^, = 

- The quadratic quantization error: \\W — VF 7 ™|| L 2 . 
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3.3 Application to the Brownian motion on L 2 ([0,T],dt) 

We present in this subsection numerical results for the above quantizer designs applied to the 
Brownian motion W on the Hilbert space (L 2 ([0, T], dt), ||-||x,2 ). 
Recall that the eigenvalues of C\y read 

^b^lTl)) 2 ' 3> - 1 

and the eigenvectors 

Uj 

which imply a regularity index of b - 




Let a be a quantizer for ®jli A/"(0, A-,), then for S 1 from (3.1) 

7 := S-'a = {t^J^J2 a i s[n ( n ^ ~ l l 2 ) t l T ) : K a 2 , . . . ) G a} (3.6) 

i>i 

provides a quantizer for W, which produces the same quantization error as a and is stationary iff 
a is. Furthermore we can restrict w.l.o.g. to the case T = 1. 

Concerning the numerical construction of a quantizer for the Brownian motion we need access 
to precomputed stationary quantizers of ®^=k i+1 J\f(0, Xj) and ®*l^ i+1 .A/'(0, 1) for all possible 
combinations of the block allocation problem. As soon as these quantizers are computed, we 
can perform the Block Allocation of the quantizer Designs to produce optimal Quantizers for 
®/ ;A'i().A,). 

For the quantizers of Design I we used the stochastic algorithm from section 3.2, whereas for 
Designs II - IV we could employ deterministic procedures for the integration on the Voronoi cells 
with max. block lengths / = 2 respectively I = 3, which provide a maximum level of stationarity, 
i.e. ||VD„|| < le~ 8 . 



n 


d n 


E||W- W^'ftf 


1 


1 


0.5000 


5 


1 


0.1271 


10 


2 


0.0921 


50 


3 


0.0558 


100 


4 


0.0475 


500 


6 


0.0353 


1000 


6 


0.0318 


5000 


8 


0.0258 


10000 


9 


0.0238 



Table 1: Quantizer Design I 



= yjsin(i/ v /A~), j>l 
= 2 for the regularly varying function 
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n 




h 


TTTV II TXT LT?7" f/ II 2 

Jtj|| w — W ' 11^2 


1 


1 






1 


0.5000 


5 


5 






1 


0.1271 


10 


10 






1 


0.0921 


50 


25 x 2 = 50 


2 + 


1 


= 3 


0.0580 


100 


50 x 2 = 50 


2 + 


1 


= 3 


0.0492 


500 


inn ^, o rnn 

100 x 2 = 500 


2 + 


1 


= 3 


0.0372 


1000 


111 x 3 x 3 = 999 


2 + 1 + 


2 


= 5 


0.0339 


5000 


166 x 10 x 3 = 4980 


2 + 2 + 


2 


= 6 


0.0276 


10000 


208 x 12 x 4 = 9984 


2 + 2 + 


2 


= 6 


0.0255 


100000 


277 x 20 x 6 x 3 = 99720 


2+2+2+ 


2 


= 8 


0.0206 



Table 2: Quantizer Design II, Z = 2 



n 


m 


k 


M\W -W^"\\ 2 T2 


1 


l 




1 


0.5000 


5 


5 




1 


0.1271 


10 


5x2 


1 + 1 


= 2 


0.0984 


50 


10 x 5 = 50 


1 + 2 


= 3 


0.0616 


100 


12 x 4 x 2 = 96 


1 + 1 + 1 


= 3 


0.0513 


500 


16 x 5 x 3 x 2 = 480 


1+1+1+1 


= 4 


0.0387 


1000 


20 x 25 x 2 = 1000 


1 + 2 + 1 


= 5 


0.0350 


5000 


26 x8x8x3 = 4992 


1+1+2+2 


= 6 


0.0285 


10000 


25 x 36 x 11 = 9900 


1 + 2 + 3 


= 6 


0.0264 


100000 


33 x 55 x 11 x 5 = 99825 


1+2+2+3 


= 8 


0.0211 



Table 3: Quantizer Design III, I = 3 



n 


m 


m 


^\\w -w^ IV \\\ 2 


1 


1 


1 


0.5000 


5 


5 


1 


0.1271 


10 


5x2 


2 


0.0984 


50 


12 x 4 = 48 


2 


0.0616 


100 


12 x 4 x 2 = 96 


3 


0.0513 


500 


16 x 5 x 3 x 2 = 480 


4 


0.0387 


1000 


23 x 7 x 3 x 2 = 966 


4 


0.0352 


5000 


26 x8x4x3x2 = 4992 


5 


0.0286 


10000 


26 x8x4x3x2x2 = 9984 


6 


0.0264 


100000 


34 x 10 x6x4x3x2x2 = 97920 


7 


0.0213 



Table 4: Quantizer Design IV 



The asymptotical performance of the quantizer designs in view of Theorem 2, i.e. 

ra^logn E||W- W^f^. 
is presented in Figure 2, where the quantization coefficient is evaluated for the Brownian Motion 
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on [0, 1] with tp(j) = ir j as 



b\b-i b 



-7T 



-2 



-=■ « 0.20264237.. 



0,25 
0,24 
0,23 
0,22 
0,21 
0,20 
0,19 
0,18 
0,17 
0,16 
0,15 



1.000 2.000 3.000 4.000 5.000 6.000 7.000 3.000 9.000 10.000 

quantizer size N 



— design I —design II with 1=2 design III with 1=3 design IV optimal Rate 



Figure 2: Asymptotics for n ^> logn x E||W — W |, ||^ 2 f° r the Designs I, II, III and IV. 



As expected we have 



E||1F-1F 7 ||| 2 < E||W- W~< f L 2 <^\\W -W"< f L 2 



and by (3.5), 



logn E||W- 



|2 <4C(1) + 1 



logn e n (W) 2 = 5.9414... logn e n {W) 2 ~ 1.2040... 



assuming C(l) = Q(l). 

Although the Designs I, II and III are asymptotically equivalent, we can observe a great supe- 
riority of Designs I and II compared to Design III. 

This is mainly caused by the better adaption to the rapidly decreasing sequence of the eigen- 
values. To give an impression of this geometrical superior adaption, we illustrate the case n = 6 in 
Figure 3. The quantizers for 0° =1 AA(0, Aj) in the figure are projected onto the first two dimen- 
sions. Within that subspace, quantizer IV is a product quantizer of a 1 x {0}, hence the rectangular 
shape of the Voronoi cells. 

As quantizer III was formerly optimized for the symmetrically distribution A^(0, 12), there are 
still to many points in the subspace generated by the eigenvector of A2, which cannot be accom- 
plished by the weightening tensor product V AW (8> a®. 

Concerning quantizer II, we see the possibly best quantizer at level 6 for (^)^ =1 ^(0, Aj), since 
the quantizer Design II produces the same quantizer for N = 6 regardless of / = 2 or I = 3 and is 
therefore equivalent to Design I. 
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Figure 3: Quantizers of size n = 6 for ®j=i -^(0> ^j) generated by Designs II, III and IV (from left 
to right) and projected on the eigenspace corresponding to Ai and A2 




0,0 0,1 0,2 0,3 0,4 0,5 o,s a? as 0,9 1,0 



t 

Figure 4: A stationary quantizer for W on L 2 ([Q, 1], dt) generated by Design I, size n = 50 and 
d* n = 3 
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0,0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1,0 

t 




3.4 Application to Riemann-Liouville processes 

We consider Riemann-Liouville processes in H = L 2 ([0,T],dt). For p G (0,oo), the Riemann- 
Liouville process X p = (Xf) tG [ 0T ] on [0, T] is denned by 

X t p := [\t-8)P-*dW s (3.7) 
Jo 

where W is a standard Brownian motion. 
Its covariance function is given by 

rsAt 

EX§X? = / {t-r)P-2( s -r)P-2dr. (3.8) 
J o 

Using p A ^-Holder continuity of the application t ^ X% from [0,T] into L 2 (P) and the Kolmorogov 
criterion one checks that X p has a pathwise continuous modification so that we may assume without 
loss of generality that X p is pathwise continuous. In particular, X p can be seen as a centered 
Gaussian random vector with values in 

H = L 2 ([0,T],dt). 

The following high-resolution formula is a consequence of a theorem by Vu and Gorenflo [19] on 
singular values of Riemann-Liouville integral operators 

Rf3 9(t) = f\t ~ sf- l g{s)ds, [3 e (0, oc). (3.9) 

For every p G (0, oo), 

e n (X p ) ~T^ +1 / 2 7r"^+^(p + l/2)^(^il) 1 / 2 r(p + l/2)(logn)^ as n -f oo. (3.10) 

2p 

This can be seen as follows. For (3 > 1/2, the Riemann-Liouville fractional integral operator 
R/3 is a bounded operator from L 2 ([0,T],dt) into L 2 ([0,T],dt). The covariance operator 

C p :L 2 ([0,T],dt)^L 2 ([0,T],^) 

of X p is given by the Fredholm transformation 

C P g(t)= [ T g(s)EX p X?ds. 
Jo 

Using (3.8), one checks that C p admits a factorization 

C p = SpS*, 

where 

Sp = T(p+l/2)R p+1 _. 

Consequently, it follows from Theorem 1 in [19] that the eigenvalues Ai > A2 > . . . > of C p satisfy 

Xj ~T 2 " +1 r(p + l/2) 2 (vrj)~ (2p+1) as j 00. (3.11) 
Now (3.10) follows from Theorem 1 (with ip(x) = T 2p+l T{p + l/2) 2 Tt- b X - b and b = 2/9+1). 
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An immediate consequence for fractionally integrated Brownian motions on [0, T] denned by 

Y ? := W)ll {t ~ s)P ~ 1Wsds (3 ' 12) 

for (3 G (0, oo) is as follows. 
For every (3 G (0, oo), 

2p -j- 1 

In fact, for p > 1/2, the Ito formula yields 



Consequently, 



x! = { P -\) j\t- s y- z *w s ds. 



Y f = -l—R [\t - sf +1 2~lw s ds = - 1 - X^ 



The assertion follows. 

One further consequence is a precise relationship between the quantization errors of Riemann- 
Liouville processes and fractional Brownian motions. The fractional Brownian motion with Hurst 
exponent p G (0, 1] is a centered pathwise continuous Gaussian process Z p = (-£f)te[o,T] having the 
covariance function 

E Z§ Z p t = ^(s 2 p + t 2 P- \s-t \ 2 P). (3.13) 



For every p G (0, 1), 



~ (r(2 P ?^Zw<i e '' m 88 °°' (3 ' 14> 



In fact, by [11], we have 



1 /2 

e n (ZP) ~ T^ +1 /V-^+5)( / 9 + 1/2)" f^T^) (r(2p + l)sin(Trp)) 1 / 2 (log n)-",n ^ oo. 

Combining this formula with (3.10) yields the assertion (3.14) 

Observe that strong equivalence e n (X p ) ~ e n (Z p ) as n — > oo is true for exactly two values of 
p G (0, 1), namely for p = 1/2 where even e n {X l l 2 ) = e n (Z 1 ^ 2 ) = e n (W) and, a bit mysterious, for 
p = 0.81557... 

The basic example (among Riemann-Liouville processes) is X 1 / 2 = W and H = L 2 ([0,T],dt), 
where 

A, = T 2 (vr(j - l/2))~ 2 , Uj (t) = ^1 sin (t/y/X^j , j > 1 (3.15) 

(see Section 3.3). 

Since for 5,pe (0, oo), 

xS+f = E£f£+J) 
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one gets expansions of X &+p from Karhunen-Loeve expansions of X p . In particular, 

oo 

However, the functions Rs(uj),j > 1, are not orthogonal in H so that the nonzero correlation 

between the components of — £(•?)) prevents the previous estimates for — X n || 2 given in 
Lemma 1 from working in this setting in the general case. 

However, when I = 1 (scalar product quantizers made up with blocks of fixed length I = 1, 
see Design I), one checks that these estimates still stand as equalities since orthogonality can now 
be substituted by the independence of £j — £j and stationarity property (2.2) of the quantizations 
£j, j > 1. It is often good enough for applications to use scalar product quantizers (see [10], [17]). 
If, for instance 6 = 1, then 

oo 

i=i 

where 

fllteXf) = - OOB(t/v^)). 

Note that ||.Ri(iij)|| 2 = T 2 ^(3 - 4(-l)^ 1 ^7i~), j > 1, where Xj = T 2 ^. Set 



3=1 

The quantization X n is non Voronoi (it is related to the Voronoi tessellation of W) and satisfies 

m 

E||X-X"f = ^TV, 2 (3-4(-ir 1 V^)e n ,(iV(0,l)) 2 + £ T^ 2 (3 - 4(-l)'" (3.16) 

j=l j>m+l 

It is possible to optimize the (scalar product) quantization error using this expression instead 
of (2.7). As concerns asymptotics, if the parameters are tuned following (2.8)-(2.10) with I = 1 and 
Xj replaced by 

Uj := T 4 fi 2 (3 + 4^777) ~ 3ir~ 4 j~ 4 as n — > oo, 

and using (3.10) gives 

(E \\X - X n \\ 2 f' 2 £ ^ UC ^ + 12 ) e n (X) as rwoo. (3.17) 

Numerical experiments seem to confirm that C(l) = Q(l). Since Q(l) = (see [5], p. 124), 

the above upper bound is then 



r- \ !/ 2 

3(6ttV3 + 1) \ 



5.02357... 
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